PCA-RECT: An Energy-efficient Object Detection Approach for Event Cameras

We present the first purely event-based, energy-efficient approach for object detection and categorization using an event camera. Compared to traditional frame-based cameras, choosing event cameras results in high temporal resolution (order of microseconds), low power consumption (few hundred mW) and wide dynamic range (120 dB) as attractive properties. However, event-based object recognition systems are far behind their frame-based counterparts in terms of accuracy. To this end, this paper presents an event-based feature extraction method devised by accumulating local activity across the image frame and then applying principal component analysis (PCA) to the normalized neighborhood region. Subsequently, we propose a backtracking-free k-d tree mechanism for efficient feature matching by taking advantage of the low-dimensionality of the feature representation. Additionally, the proposed k-d tree mechanism allows for feature selection to obtain a lower-dimensional dictionary representation when hardware resources are limited to implement dimensionality reduction. Consequently, the proposed system can be realized on a field-programmable gate array (FPGA) device leading to high performance over resource ratio. The proposed system is tested on real-world event-based datasets for object categorization, showing superior classification performance and relevance to state-of-the-art algorithms. Additionally, we verified the object detection method and real-time FPGA performance in lab settings under non-controlled illumination conditions with limited training data and ground truth annotations.Comment: Accepted in ACCV 2018 Workshops, to appea

A 5 Meps $100 USB2.0 Address-Event Monitor-Sequencer Interface

This paper describes a high-speed USB2.0 address-event representation (AER) interface that allows simultaneous monitoring and sequencing of precisely timed AER data. This low-cost (<$100), two chip, bus powered interface can achieve sustained AER event rates of 5 megaevents per second (Meps). Several boards can be electrically synchronized, allowing simultaneous synchronized capture from multiple devices. It has three parallel AER ports, one for sequencing, one for monitoring and one for passing through the monitored events. This paper also describes the host software infrastructure that makes the board usable for a heterogeneous mixture of AER devices and that allows recording and playback of recorded data

Measuring diameters and velocities of artificial raindrops with a neuromorphic event camera

Hydrometers that measure size and velocity distributions of precipitation are needed for research and corrections of rainfall estimates from weather radars and microwave links. Existing optical disdrometers measure droplet size distributions, but underestimate small raindrops and are impractical for widespread always-on IoT deployment. We study the feasibility of measuring droplet size and velocity using a neuromorphic event camera. These dynamic vision sensors asynchronously output a sparse stream of pixel brightness changes. Droplets falling through the plane of focus of a steeply down-looking camera create events generated by the motion of the droplet across the field of view. Droplet size and speed are inferred from the hourglass-shaped stream of events. Using an improved hard disk arm actuator to reliably generate artificial raindrops with a range of small sizes, our experiments show maximum errors of 7 % (mean absolute percentage error) for droplet sizes from 0.3 to 2.5 mm and speeds from 1.3 to 8.0 m s−1. Measurements with the same setup from a commercial PARSIVEL disdrometer show similar results. Both devices slightly overestimate the small droplet volume with a volume overestimation of 25 % from the event camera measurements and 50 % from the PARSIVEL instrument. Each droplet requires processing of 5000 to 50 000 brightness change events, potentially enabling low-power always-on disdrometers that consume power proportional to the rainfall rate. Data and code are available at the paper website https://sites.google.com/view/dvs-disdrometer/home (Micev et al., 2023).</p

Knot localization in adsorbing polymer rings

We study by Monte Carlo simulations a model of knotted polymer ring adsorbing onto an impenetrable, attractive wall. The polymer is described by a self-avoiding polygon (SAP) on the cubic lattice. We find that the adsorption transition temperature, the crossover exponent $\phi$ and the metric exponent $\nu$, are the same as in the model where the topology of the ring is unrestricted. By measuring the average length of the knotted portion of the ring we are able to show that adsorbed knots are localized. This knot localization transition is triggered by the adsorption transition but is accompanied by a less sharp variation of the exponent related to the degree of localization. Indeed, for a whole interval below the adsorption transition, one can not exclude a contiuous variation with temperature of this exponent. Deep into the adsorbed phase we are able to verify that knot localization is strong and well described in terms of the flat knot model.Comment: 27 pages, 10 figures. Submitter to Phys. Rev.

Confession Session: Learning from Others Mistakes

Accepted versio

Live demonstration: Gesture-Based remote control using stereo pair of dynamic vision sensors

This demonstration shows a natural gesture interface for console entertainment devices using as input a stereo pair of dynamic vision sensors. The event-based processing of the sparse sensor output allows fluid interaction at a laptop processor load of less than 3%

Functional and Biogenetical Heterogeneity of the Inner Membrane of Rat-Liver Mitochondria

Rat liver mitochondria were fragmented by a combined technique of swelling, shrinking, and sonication. Fragments of inner membrane were separated by density gradient centrifugation. They differed in several respects: electronmicroscopic appearance, phospholipid and cytochrome contents, electrophoretic behaviour of proteins and enzymatic activities. Three types of inner membrane fractions were isolated. The first type is characterized by a high activity of metal chelatase, low activities of succinate-cytochrome c reductase and of glycerolphosphate dehydrogenase, as well as by a high phospholipid content and low contents of cytochromes aa3 and b. The second type displays maximal activities of glycerolphosphate dehydrogenase and metal chelatase, but contains relatively little cytochromes and has low succinate-cytochrome c reductase activity. The third type exhibits highest succinate-cytochrome c reductase activity, a high metal chelatase activity and highest cytochrome contents. However, this fraction was low in both glycerolphosphate dehydrogenase activity and phospholipid content. This fraction was also richest in the following enzyme activities: cytochrome oxidase, oligomycin-sensitive ATPase, proline oxidase, 3-hydroxybutyrate dehydrogenase and rotenone-sensitive NADH-cytochrome c reductase. Amino acid incorporation in vitro and in vivo in the presence of cycloheximide occurs predominantly into inner membrane fractions from the second type. These data suggest that the inner membrane is composed of differently organized parts, and that polypeptides synthesized by mitochondrial ribosomes are integrated into specific parts of the inner membrane

Semi-Dense 3D Reconstruction with a Stereo Event Camera

Event cameras are bio-inspired sensors that offer several advantages, such as low latency, high-speed and high dynamic range, to tackle challenging scenarios in computer vision. This paper presents a solution to the problem of 3D reconstruction from data captured by a stereo event-camera rig moving in a static scene, such as in the context of stereo Simultaneous Localization and Mapping. The proposed method consists of the optimization of an energy function designed to exploit small-baseline spatio-temporal consistency of events triggered across both stereo image planes. To improve the density of the reconstruction and to reduce the uncertainty of the estimation, a probabilistic depth-fusion strategy is also developed. The resulting method has no special requirements on either the motion of the stereo event-camera rig or on prior knowledge about the scene. Experiments demonstrate our method can deal with both texture-rich scenes as well as sparse scenes, outperforming state-of-the-art stereo methods based on event data image representations.Comment: 19 pages, 8 figures, Video: https://youtu.be/Qrnpj2FD1e